{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature extraction with tsfresh transformer\n",
    "\n",
    "In this tutorial, we show how you can use sktime with [tsfresh](https://tsfresh.readthedocs.io) to first extract features from time series, so that we can then use any scikit-learn estimator.\n",
    "\n",
    "## Preliminaries\n",
    "You have to install tsfresh if you haven't already. To install it, uncomment the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:39.713903Z",
     "iopub.status.busy": "2020-12-19T14:30:39.713342Z",
     "iopub.status.idle": "2020-12-19T14:30:39.715128Z",
     "shell.execute_reply": "2020-12-19T14:30:39.715641Z"
    }
   },
   "outputs": [],
   "source": [
    "# !pip install --upgrade tsfresh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:39.719083Z",
     "iopub.status.busy": "2020-12-19T14:30:39.718586Z",
     "iopub.status.idle": "2020-12-19T14:30:40.743724Z",
     "shell.execute_reply": "2020-12-19T14:30:40.744213Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "from sktime.datasets import load_arrow_head, load_basic_motions\n",
    "from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Univariate time series classification data\n",
    "\n",
    "For more details on the data set, see the [univariate time series classification notebook](https://github.com/alan-turing-institute/sktime/blob/master/examples/02_classification_univariate.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:40.748159Z",
     "iopub.status.busy": "2020-12-19T14:30:40.747656Z",
     "iopub.status.idle": "2020-12-19T14:30:40.795200Z",
     "shell.execute_reply": "2020-12-19T14:30:40.795889Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(158, 1) (158,) (53, 1) (53,)\n"
     ]
    }
   ],
   "source": [
    "X, y = load_arrow_head(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:40.808841Z",
     "iopub.status.busy": "2020-12-19T14:30:40.808198Z",
     "iopub.status.idle": "2020-12-19T14:30:40.816155Z",
     "shell.execute_reply": "2020-12-19T14:30:40.816682Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dim_0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>0     -1.7390\n",
       "1     -1.7415\n",
       "2     -1.7329\n",
       "3   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0     -2.1788\n",
       "1     -2.1751\n",
       "2     -2.1550\n",
       "3   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>0     -1.7427\n",
       "1     -1.7399\n",
       "2     -1.7102\n",
       "3   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>0     -1.9969\n",
       "1     -2.0076\n",
       "2     -2.0010\n",
       "3   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>0     -1.8284\n",
       "1     -1.8393\n",
       "2     -1.8025\n",
       "3   ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 dim_0\n",
       "173  0     -1.7390\n",
       "1     -1.7415\n",
       "2     -1.7329\n",
       "3   ...\n",
       "17   0     -2.1788\n",
       "1     -2.1751\n",
       "2     -2.1550\n",
       "3   ...\n",
       "145  0     -1.7427\n",
       "1     -1.7399\n",
       "2     -1.7102\n",
       "3   ...\n",
       "59   0     -1.9969\n",
       "1     -2.0076\n",
       "2     -2.0010\n",
       "3   ...\n",
       "95   0     -1.8284\n",
       "1     -1.8393\n",
       "2     -1.8025\n",
       "3   ..."
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:40.820002Z",
     "iopub.status.busy": "2020-12-19T14:30:40.819515Z",
     "iopub.status.idle": "2020-12-19T14:30:40.821979Z",
     "shell.execute_reply": "2020-12-19T14:30:40.822517Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['0', '1', '2'], dtype=object)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  binary classification task\n",
    "np.unique(y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using tsfresh to extract features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:40.829452Z",
     "iopub.status.busy": "2020-12-19T14:30:40.828907Z",
     "iopub.status.idle": "2020-12-19T14:30:53.049755Z",
     "shell.execute_reply": "2020-12-19T14:30:53.050249Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mloning/Documents/Research/software/sktime/sktime/sktime/transformations/panel/tsfresh.py:164: UserWarning: tsfresh requires a unique index, but found non-unique. To avoid this warning, please make sure the index of X contains only unique values.\n",
      "  \"tsfresh requires a unique index, but found \"\n",
      "Feature Extraction: 100%|██████████| 5/5 [00:11<00:00,  2.32s/it]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dim_0__variance_larger_than_standard_deviation</th>\n",
       "      <th>dim_0__has_duplicate_max</th>\n",
       "      <th>dim_0__has_duplicate_min</th>\n",
       "      <th>dim_0__has_duplicate</th>\n",
       "      <th>dim_0__sum_values</th>\n",
       "      <th>dim_0__abs_energy</th>\n",
       "      <th>dim_0__mean_abs_change</th>\n",
       "      <th>dim_0__mean_change</th>\n",
       "      <th>dim_0__mean_second_derivative_central</th>\n",
       "      <th>dim_0__median</th>\n",
       "      <th>...</th>\n",
       "      <th>dim_0__fourier_entropy__bins_2</th>\n",
       "      <th>dim_0__fourier_entropy__bins_3</th>\n",
       "      <th>dim_0__fourier_entropy__bins_5</th>\n",
       "      <th>dim_0__fourier_entropy__bins_10</th>\n",
       "      <th>dim_0__fourier_entropy__bins_100</th>\n",
       "      <th>dim_0__permutation_entropy__dimension_3__tau_1</th>\n",
       "      <th>dim_0__permutation_entropy__dimension_4__tau_1</th>\n",
       "      <th>dim_0__permutation_entropy__dimension_5__tau_1</th>\n",
       "      <th>dim_0__permutation_entropy__dimension_6__tau_1</th>\n",
       "      <th>dim_0__permutation_entropy__dimension_7__tau_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.000161</td>\n",
       "      <td>250.000831</td>\n",
       "      <td>0.315017</td>\n",
       "      <td>0.005540</td>\n",
       "      <td>-0.000152</td>\n",
       "      <td>0.166000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.08151</td>\n",
       "      <td>0.092513</td>\n",
       "      <td>0.173767</td>\n",
       "      <td>0.219798</td>\n",
       "      <td>1.219806</td>\n",
       "      <td>1.447748</td>\n",
       "      <td>2.089695</td>\n",
       "      <td>2.619112</td>\n",
       "      <td>3.055134</td>\n",
       "      <td>3.411670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.000050</td>\n",
       "      <td>250.000350</td>\n",
       "      <td>0.351749</td>\n",
       "      <td>0.004856</td>\n",
       "      <td>-0.000231</td>\n",
       "      <td>-0.057820</td>\n",
       "      <td>...</td>\n",
       "      <td>0.08151</td>\n",
       "      <td>0.081510</td>\n",
       "      <td>0.138673</td>\n",
       "      <td>0.250609</td>\n",
       "      <td>1.340724</td>\n",
       "      <td>1.568692</td>\n",
       "      <td>2.482612</td>\n",
       "      <td>3.225589</td>\n",
       "      <td>3.789130</td>\n",
       "      <td>4.198932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.000655</td>\n",
       "      <td>249.999869</td>\n",
       "      <td>0.323647</td>\n",
       "      <td>0.005567</td>\n",
       "      <td>-0.000038</td>\n",
       "      <td>0.176560</td>\n",
       "      <td>...</td>\n",
       "      <td>0.08151</td>\n",
       "      <td>0.092513</td>\n",
       "      <td>0.173767</td>\n",
       "      <td>0.285506</td>\n",
       "      <td>1.292960</td>\n",
       "      <td>1.439692</td>\n",
       "      <td>2.121711</td>\n",
       "      <td>2.705458</td>\n",
       "      <td>3.143189</td>\n",
       "      <td>3.489565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.000223</td>\n",
       "      <td>249.999003</td>\n",
       "      <td>0.341776</td>\n",
       "      <td>0.004884</td>\n",
       "      <td>0.000024</td>\n",
       "      <td>-0.194770</td>\n",
       "      <td>...</td>\n",
       "      <td>0.08151</td>\n",
       "      <td>0.081510</td>\n",
       "      <td>0.127671</td>\n",
       "      <td>0.184769</td>\n",
       "      <td>1.226987</td>\n",
       "      <td>1.535460</td>\n",
       "      <td>2.355170</td>\n",
       "      <td>2.990719</td>\n",
       "      <td>3.530660</td>\n",
       "      <td>3.961005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.000167</td>\n",
       "      <td>250.001244</td>\n",
       "      <td>0.365629</td>\n",
       "      <td>0.006513</td>\n",
       "      <td>-0.000238</td>\n",
       "      <td>0.009969</td>\n",
       "      <td>...</td>\n",
       "      <td>0.08151</td>\n",
       "      <td>0.081510</td>\n",
       "      <td>0.092513</td>\n",
       "      <td>0.173767</td>\n",
       "      <td>1.159755</td>\n",
       "      <td>1.568399</td>\n",
       "      <td>2.464715</td>\n",
       "      <td>3.270374</td>\n",
       "      <td>3.890812</td>\n",
       "      <td>4.321230</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 773 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   dim_0__variance_larger_than_standard_deviation  dim_0__has_duplicate_max  \\\n",
       "0                                             0.0                       1.0   \n",
       "1                                             0.0                       0.0   \n",
       "2                                             0.0                       0.0   \n",
       "3                                             0.0                       0.0   \n",
       "4                                             0.0                       0.0   \n",
       "\n",
       "   dim_0__has_duplicate_min  dim_0__has_duplicate  dim_0__sum_values  \\\n",
       "0                       0.0                   1.0           0.000161   \n",
       "1                       0.0                   1.0          -0.000050   \n",
       "2                       0.0                   1.0           0.000655   \n",
       "3                       0.0                   1.0          -0.000223   \n",
       "4                       0.0                   1.0          -0.000167   \n",
       "\n",
       "   dim_0__abs_energy  dim_0__mean_abs_change  dim_0__mean_change  \\\n",
       "0         250.000831                0.315017            0.005540   \n",
       "1         250.000350                0.351749            0.004856   \n",
       "2         249.999869                0.323647            0.005567   \n",
       "3         249.999003                0.341776            0.004884   \n",
       "4         250.001244                0.365629            0.006513   \n",
       "\n",
       "   dim_0__mean_second_derivative_central  dim_0__median  ...  \\\n",
       "0                              -0.000152       0.166000  ...   \n",
       "1                              -0.000231      -0.057820  ...   \n",
       "2                              -0.000038       0.176560  ...   \n",
       "3                               0.000024      -0.194770  ...   \n",
       "4                              -0.000238       0.009969  ...   \n",
       "\n",
       "   dim_0__fourier_entropy__bins_2  dim_0__fourier_entropy__bins_3  \\\n",
       "0                         0.08151                        0.092513   \n",
       "1                         0.08151                        0.081510   \n",
       "2                         0.08151                        0.092513   \n",
       "3                         0.08151                        0.081510   \n",
       "4                         0.08151                        0.081510   \n",
       "\n",
       "   dim_0__fourier_entropy__bins_5  dim_0__fourier_entropy__bins_10  \\\n",
       "0                        0.173767                         0.219798   \n",
       "1                        0.138673                         0.250609   \n",
       "2                        0.173767                         0.285506   \n",
       "3                        0.127671                         0.184769   \n",
       "4                        0.092513                         0.173767   \n",
       "\n",
       "   dim_0__fourier_entropy__bins_100  \\\n",
       "0                          1.219806   \n",
       "1                          1.340724   \n",
       "2                          1.292960   \n",
       "3                          1.226987   \n",
       "4                          1.159755   \n",
       "\n",
       "   dim_0__permutation_entropy__dimension_3__tau_1  \\\n",
       "0                                        1.447748   \n",
       "1                                        1.568692   \n",
       "2                                        1.439692   \n",
       "3                                        1.535460   \n",
       "4                                        1.568399   \n",
       "\n",
       "   dim_0__permutation_entropy__dimension_4__tau_1  \\\n",
       "0                                        2.089695   \n",
       "1                                        2.482612   \n",
       "2                                        2.121711   \n",
       "3                                        2.355170   \n",
       "4                                        2.464715   \n",
       "\n",
       "   dim_0__permutation_entropy__dimension_5__tau_1  \\\n",
       "0                                        2.619112   \n",
       "1                                        3.225589   \n",
       "2                                        2.705458   \n",
       "3                                        2.990719   \n",
       "4                                        3.270374   \n",
       "\n",
       "   dim_0__permutation_entropy__dimension_6__tau_1  \\\n",
       "0                                        3.055134   \n",
       "1                                        3.789130   \n",
       "2                                        3.143189   \n",
       "3                                        3.530660   \n",
       "4                                        3.890812   \n",
       "\n",
       "   dim_0__permutation_entropy__dimension_7__tau_1  \n",
       "0                                        3.411670  \n",
       "1                                        4.198932  \n",
       "2                                        3.489565  \n",
       "3                                        3.961005  \n",
       "4                                        4.321230  \n",
       "\n",
       "[5 rows x 773 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# tf = TsFreshTransformer()\n",
    "t = TSFreshFeatureExtractor(default_fc_parameters=\"efficient\", show_warnings=False)\n",
    "Xt = t.fit_transform(X_train)\n",
    "Xt.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using tsfresh with sktime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:30:53.062147Z",
     "iopub.status.busy": "2020-12-19T14:30:53.061631Z",
     "iopub.status.idle": "2020-12-19T14:31:09.307275Z",
     "shell.execute_reply": "2020-12-19T14:31:09.307781Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mloning/Documents/Research/software/sktime/sktime/sktime/transformations/panel/tsfresh.py:164: UserWarning: tsfresh requires a unique index, but found non-unique. To avoid this warning, please make sure the index of X contains only unique values.\n",
      "  \"tsfresh requires a unique index, but found \"\n",
      "Feature Extraction: 100%|██████████| 5/5 [00:11<00:00,  2.26s/it]\n",
      "Feature Extraction: 100%|██████████| 5/5 [00:03<00:00,  1.30it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7358490566037735"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier = make_pipeline(\n",
    "    TSFreshFeatureExtractor(default_fc_parameters=\"efficient\", show_warnings=False),\n",
    "    RandomForestClassifier(),\n",
    ")\n",
    "classifier.fit(X_train, y_train)\n",
    "classifier.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multivariate time series classification data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:31:09.311742Z",
     "iopub.status.busy": "2020-12-19T14:31:09.311092Z",
     "iopub.status.idle": "2020-12-19T14:31:09.380791Z",
     "shell.execute_reply": "2020-12-19T14:31:09.381304Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60, 6) (60,) (20, 6) (20,)\n"
     ]
    }
   ],
   "source": [
    "X, y = load_basic_motions(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:31:09.425476Z",
     "iopub.status.busy": "2020-12-19T14:31:09.424972Z",
     "iopub.status.idle": "2020-12-19T14:31:09.427185Z",
     "shell.execute_reply": "2020-12-19T14:31:09.427741Z"
    },
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dim_0</th>\n",
       "      <th>dim_1</th>\n",
       "      <th>dim_2</th>\n",
       "      <th>dim_3</th>\n",
       "      <th>dim_4</th>\n",
       "      <th>dim_5</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0     0.648833\n",
       "1     0.648833\n",
       "2     0.076985\n",
       "3...</td>\n",
       "      <td>0    -0.996722\n",
       "1    -0.996722\n",
       "2    -0.897264\n",
       "3...</td>\n",
       "      <td>0    -0.644136\n",
       "1    -0.644136\n",
       "2     0.970515\n",
       "3...</td>\n",
       "      <td>0    -0.101208\n",
       "1    -0.101208\n",
       "2    -0.407496\n",
       "3...</td>\n",
       "      <td>0     0.055931\n",
       "1     0.055931\n",
       "2    -0.157139\n",
       "3...</td>\n",
       "      <td>0    -0.031960\n",
       "1    -0.031960\n",
       "2    -0.343575\n",
       "3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0      1.463566\n",
       "1      1.463566\n",
       "2      6.16934...</td>\n",
       "      <td>0      1.782945\n",
       "1      1.782945\n",
       "2      8.09897...</td>\n",
       "      <td>0    -0.817491\n",
       "1    -0.817491\n",
       "2    -5.628303\n",
       "3...</td>\n",
       "      <td>0     0.082565\n",
       "1     0.082565\n",
       "2    -2.671363\n",
       "3...</td>\n",
       "      <td>0     0.159802\n",
       "1     0.159802\n",
       "2     0.282318\n",
       "3...</td>\n",
       "      <td>0     0.095881\n",
       "1     0.095881\n",
       "2    -1.502142\n",
       "3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0      3.789469\n",
       "1      3.789469\n",
       "2      1.78594...</td>\n",
       "      <td>0     -1.353556\n",
       "1     -1.353556\n",
       "2    -10.69460...</td>\n",
       "      <td>0    -0.685072\n",
       "1    -0.685072\n",
       "2    -4.465480\n",
       "3...</td>\n",
       "      <td>0    -0.021307\n",
       "1    -0.021307\n",
       "2     2.753927\n",
       "3...</td>\n",
       "      <td>0    -0.159802\n",
       "1    -0.159802\n",
       "2    -0.820319\n",
       "3...</td>\n",
       "      <td>0     0.133169\n",
       "1     0.133169\n",
       "2     2.974987\n",
       "3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0    -0.761604\n",
       "1    -0.761604\n",
       "2     0.121078\n",
       "3...</td>\n",
       "      <td>0     0.260125\n",
       "1     0.260125\n",
       "2    -1.423255\n",
       "3...</td>\n",
       "      <td>0    -0.064487\n",
       "1    -0.064487\n",
       "2     0.075600\n",
       "3...</td>\n",
       "      <td>0     0.069248\n",
       "1     0.069248\n",
       "2    -0.282318\n",
       "3...</td>\n",
       "      <td>0     0.242367\n",
       "1     0.242367\n",
       "2    -0.332922\n",
       "3...</td>\n",
       "      <td>0    -0.007990\n",
       "1    -0.007990\n",
       "2     0.239704\n",
       "3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0     -0.193013\n",
       "1     -0.193013\n",
       "2      2.40398...</td>\n",
       "      <td>0     -0.106266\n",
       "1     -0.106266\n",
       "2      0.52392...</td>\n",
       "      <td>0    -0.636563\n",
       "1    -0.636563\n",
       "2    -1.166243\n",
       "3...</td>\n",
       "      <td>0    -0.087891\n",
       "1    -0.087891\n",
       "2    -2.716640\n",
       "3...</td>\n",
       "      <td>0     0.010653\n",
       "1     0.010653\n",
       "2     1.297062\n",
       "3...</td>\n",
       "      <td>0     0.205080\n",
       "1     0.205080\n",
       "2    -0.609912\n",
       "3...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                dim_0  \\\n",
       "21  0     0.648833\n",
       "1     0.648833\n",
       "2     0.076985\n",
       "3...   \n",
       "13  0      1.463566\n",
       "1      1.463566\n",
       "2      6.16934...   \n",
       "17  0      3.789469\n",
       "1      3.789469\n",
       "2      1.78594...   \n",
       "26  0    -0.761604\n",
       "1    -0.761604\n",
       "2     0.121078\n",
       "3...   \n",
       "11  0     -0.193013\n",
       "1     -0.193013\n",
       "2      2.40398...   \n",
       "\n",
       "                                                dim_1  \\\n",
       "21  0    -0.996722\n",
       "1    -0.996722\n",
       "2    -0.897264\n",
       "3...   \n",
       "13  0      1.782945\n",
       "1      1.782945\n",
       "2      8.09897...   \n",
       "17  0     -1.353556\n",
       "1     -1.353556\n",
       "2    -10.69460...   \n",
       "26  0     0.260125\n",
       "1     0.260125\n",
       "2    -1.423255\n",
       "3...   \n",
       "11  0     -0.106266\n",
       "1     -0.106266\n",
       "2      0.52392...   \n",
       "\n",
       "                                                dim_2  \\\n",
       "21  0    -0.644136\n",
       "1    -0.644136\n",
       "2     0.970515\n",
       "3...   \n",
       "13  0    -0.817491\n",
       "1    -0.817491\n",
       "2    -5.628303\n",
       "3...   \n",
       "17  0    -0.685072\n",
       "1    -0.685072\n",
       "2    -4.465480\n",
       "3...   \n",
       "26  0    -0.064487\n",
       "1    -0.064487\n",
       "2     0.075600\n",
       "3...   \n",
       "11  0    -0.636563\n",
       "1    -0.636563\n",
       "2    -1.166243\n",
       "3...   \n",
       "\n",
       "                                                dim_3  \\\n",
       "21  0    -0.101208\n",
       "1    -0.101208\n",
       "2    -0.407496\n",
       "3...   \n",
       "13  0     0.082565\n",
       "1     0.082565\n",
       "2    -2.671363\n",
       "3...   \n",
       "17  0    -0.021307\n",
       "1    -0.021307\n",
       "2     2.753927\n",
       "3...   \n",
       "26  0     0.069248\n",
       "1     0.069248\n",
       "2    -0.282318\n",
       "3...   \n",
       "11  0    -0.087891\n",
       "1    -0.087891\n",
       "2    -2.716640\n",
       "3...   \n",
       "\n",
       "                                                dim_4  \\\n",
       "21  0     0.055931\n",
       "1     0.055931\n",
       "2    -0.157139\n",
       "3...   \n",
       "13  0     0.159802\n",
       "1     0.159802\n",
       "2     0.282318\n",
       "3...   \n",
       "17  0    -0.159802\n",
       "1    -0.159802\n",
       "2    -0.820319\n",
       "3...   \n",
       "26  0     0.242367\n",
       "1     0.242367\n",
       "2    -0.332922\n",
       "3...   \n",
       "11  0     0.010653\n",
       "1     0.010653\n",
       "2     1.297062\n",
       "3...   \n",
       "\n",
       "                                                dim_5  \n",
       "21  0    -0.031960\n",
       "1    -0.031960\n",
       "2    -0.343575\n",
       "3...  \n",
       "13  0     0.095881\n",
       "1     0.095881\n",
       "2    -1.502142\n",
       "3...  \n",
       "17  0     0.133169\n",
       "1     0.133169\n",
       "2     2.974987\n",
       "3...  \n",
       "26  0    -0.007990\n",
       "1    -0.007990\n",
       "2     0.239704\n",
       "3...  \n",
       "11  0     0.205080\n",
       "1     0.205080\n",
       "2    -0.609912\n",
       "3...  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  multivariate input data\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:31:09.516548Z",
     "iopub.status.busy": "2020-12-19T14:31:09.515810Z",
     "iopub.status.idle": "2020-12-19T14:31:32.787406Z",
     "shell.execute_reply": "2020-12-19T14:31:32.788316Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mloning/Documents/Research/software/sktime/sktime/sktime/transformations/panel/tsfresh.py:164: UserWarning: tsfresh requires a unique index, but found non-unique. To avoid this warning, please make sure the index of X contains only unique values.\n",
      "  \"tsfresh requires a unique index, but found \"\n",
      "Feature Extraction: 100%|██████████| 5/5 [00:21<00:00,  4.24s/it]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dim_0__variance_larger_than_standard_deviation</th>\n",
       "      <th>dim_0__has_duplicate_max</th>\n",
       "      <th>dim_0__has_duplicate_min</th>\n",
       "      <th>dim_0__has_duplicate</th>\n",
       "      <th>dim_0__sum_values</th>\n",
       "      <th>dim_0__abs_energy</th>\n",
       "      <th>dim_0__mean_abs_change</th>\n",
       "      <th>dim_0__mean_change</th>\n",
       "      <th>dim_0__mean_second_derivative_central</th>\n",
       "      <th>dim_0__median</th>\n",
       "      <th>...</th>\n",
       "      <th>dim_5__fourier_entropy__bins_2</th>\n",
       "      <th>dim_5__fourier_entropy__bins_3</th>\n",
       "      <th>dim_5__fourier_entropy__bins_5</th>\n",
       "      <th>dim_5__fourier_entropy__bins_10</th>\n",
       "      <th>dim_5__fourier_entropy__bins_100</th>\n",
       "      <th>dim_5__permutation_entropy__dimension_3__tau_1</th>\n",
       "      <th>dim_5__permutation_entropy__dimension_4__tau_1</th>\n",
       "      <th>dim_5__permutation_entropy__dimension_5__tau_1</th>\n",
       "      <th>dim_5__permutation_entropy__dimension_6__tau_1</th>\n",
       "      <th>dim_5__permutation_entropy__dimension_7__tau_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.045746</td>\n",
       "      <td>172.027276</td>\n",
       "      <td>0.807892</td>\n",
       "      <td>0.001584</td>\n",
       "      <td>0.003131</td>\n",
       "      <td>0.422100</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>1.241657</td>\n",
       "      <td>1.494736</td>\n",
       "      <td>2.333086</td>\n",
       "      <td>3.047524</td>\n",
       "      <td>3.577109</td>\n",
       "      <td>3.928619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>575.369181</td>\n",
       "      <td>18681.476663</td>\n",
       "      <td>9.290810</td>\n",
       "      <td>-0.150685</td>\n",
       "      <td>-0.004437</td>\n",
       "      <td>11.136336</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>1.343990</td>\n",
       "      <td>1.540222</td>\n",
       "      <td>2.478743</td>\n",
       "      <td>3.332544</td>\n",
       "      <td>3.891606</td>\n",
       "      <td>4.245651</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>456.363177</td>\n",
       "      <td>14668.442452</td>\n",
       "      <td>8.609941</td>\n",
       "      <td>-0.103845</td>\n",
       "      <td>0.003627</td>\n",
       "      <td>10.290202</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.356468</td>\n",
       "      <td>1.923853</td>\n",
       "      <td>1.538814</td>\n",
       "      <td>2.523494</td>\n",
       "      <td>3.444948</td>\n",
       "      <td>4.027225</td>\n",
       "      <td>4.375502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>73.888480</td>\n",
       "      <td>220.949429</td>\n",
       "      <td>1.057349</td>\n",
       "      <td>-0.002087</td>\n",
       "      <td>-0.003908</td>\n",
       "      <td>0.613719</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>1.064807</td>\n",
       "      <td>1.530752</td>\n",
       "      <td>2.427612</td>\n",
       "      <td>3.185985</td>\n",
       "      <td>3.780048</td>\n",
       "      <td>4.133971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>450.805329</td>\n",
       "      <td>13280.972257</td>\n",
       "      <td>7.340858</td>\n",
       "      <td>-0.079199</td>\n",
       "      <td>0.020523</td>\n",
       "      <td>9.844745</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165443</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.192626</td>\n",
       "      <td>0.288342</td>\n",
       "      <td>1.524120</td>\n",
       "      <td>1.597675</td>\n",
       "      <td>2.690962</td>\n",
       "      <td>3.511981</td>\n",
       "      <td>4.075170</td>\n",
       "      <td>4.366321</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 4638 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   dim_0__variance_larger_than_standard_deviation  dim_0__has_duplicate_max  \\\n",
       "0                                             1.0                       1.0   \n",
       "1                                             1.0                       1.0   \n",
       "2                                             1.0                       0.0   \n",
       "3                                             1.0                       0.0   \n",
       "4                                             1.0                       0.0   \n",
       "\n",
       "   dim_0__has_duplicate_min  dim_0__has_duplicate  dim_0__sum_values  \\\n",
       "0                       0.0                   1.0          57.045746   \n",
       "1                       0.0                   1.0         575.369181   \n",
       "2                       0.0                   1.0         456.363177   \n",
       "3                       0.0                   1.0          73.888480   \n",
       "4                       1.0                   1.0         450.805329   \n",
       "\n",
       "   dim_0__abs_energy  dim_0__mean_abs_change  dim_0__mean_change  \\\n",
       "0         172.027276                0.807892            0.001584   \n",
       "1       18681.476663                9.290810           -0.150685   \n",
       "2       14668.442452                8.609941           -0.103845   \n",
       "3         220.949429                1.057349           -0.002087   \n",
       "4       13280.972257                7.340858           -0.079199   \n",
       "\n",
       "   dim_0__mean_second_derivative_central  dim_0__median  ...  \\\n",
       "0                               0.003131       0.422100  ...   \n",
       "1                              -0.004437      11.136336  ...   \n",
       "2                               0.003627      10.290202  ...   \n",
       "3                              -0.003908       0.613719  ...   \n",
       "4                               0.020523       9.844745  ...   \n",
       "\n",
       "   dim_5__fourier_entropy__bins_2  dim_5__fourier_entropy__bins_3  \\\n",
       "0                        0.165443                        0.165443   \n",
       "1                        0.165443                        0.165443   \n",
       "2                        0.165443                        0.192626   \n",
       "3                        0.165443                        0.192626   \n",
       "4                        0.165443                        0.192626   \n",
       "\n",
       "   dim_5__fourier_entropy__bins_5  dim_5__fourier_entropy__bins_10  \\\n",
       "0                        0.165443                         0.165443   \n",
       "1                        0.165443                         0.192626   \n",
       "2                        0.192626                         0.356468   \n",
       "3                        0.192626                         0.192626   \n",
       "4                        0.192626                         0.288342   \n",
       "\n",
       "   dim_5__fourier_entropy__bins_100  \\\n",
       "0                          1.241657   \n",
       "1                          1.343990   \n",
       "2                          1.923853   \n",
       "3                          1.064807   \n",
       "4                          1.524120   \n",
       "\n",
       "   dim_5__permutation_entropy__dimension_3__tau_1  \\\n",
       "0                                        1.494736   \n",
       "1                                        1.540222   \n",
       "2                                        1.538814   \n",
       "3                                        1.530752   \n",
       "4                                        1.597675   \n",
       "\n",
       "   dim_5__permutation_entropy__dimension_4__tau_1  \\\n",
       "0                                        2.333086   \n",
       "1                                        2.478743   \n",
       "2                                        2.523494   \n",
       "3                                        2.427612   \n",
       "4                                        2.690962   \n",
       "\n",
       "   dim_5__permutation_entropy__dimension_5__tau_1  \\\n",
       "0                                        3.047524   \n",
       "1                                        3.332544   \n",
       "2                                        3.444948   \n",
       "3                                        3.185985   \n",
       "4                                        3.511981   \n",
       "\n",
       "   dim_5__permutation_entropy__dimension_6__tau_1  \\\n",
       "0                                        3.577109   \n",
       "1                                        3.891606   \n",
       "2                                        4.027225   \n",
       "3                                        3.780048   \n",
       "4                                        4.075170   \n",
       "\n",
       "   dim_5__permutation_entropy__dimension_7__tau_1  \n",
       "0                                        3.928619  \n",
       "1                                        4.245651  \n",
       "2                                        4.375502  \n",
       "3                                        4.133971  \n",
       "4                                        4.366321  \n",
       "\n",
       "[5 rows x 4638 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = TSFreshFeatureExtractor(default_fc_parameters=\"efficient\", show_warnings=False)\n",
    "Xt = t.fit_transform(X_train)\n",
    "Xt.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using tsfresh for forecasting\n",
    "You can also use tsfresh to do univariate forecasting. To find out more about forecasting, check out our forecasting tutorial notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:31:32.796083Z",
     "iopub.status.busy": "2020-12-19T14:31:32.795215Z",
     "iopub.status.idle": "2020-12-19T14:31:49.386345Z",
     "shell.execute_reply": "2020-12-19T14:31:49.386917Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "from sktime.datasets import load_airline\n",
    "from sktime.forecasting.base import ForecastingHorizon\n",
    "from sktime.forecasting.compose import ReducedTimeSeriesRegressionForecaster\n",
    "from sktime.forecasting.model_selection import temporal_train_test_split\n",
    "\n",
    "y = load_airline()\n",
    "y_train, y_test = temporal_train_test_split(y)\n",
    "\n",
    "regressor = make_pipeline(\n",
    "    TSFreshFeatureExtractor(show_warnings=False, disable_progressbar=True),\n",
    "    RandomForestRegressor(),\n",
    ")\n",
    "forecaster = ReducedTimeSeriesRegressionForecaster(regressor, window_length=12)\n",
    "forecaster.fit(y_train)\n",
    "\n",
    "fh = ForecastingHorizon(y_test.index, is_relative=False)\n",
    "y_pred = forecaster.predict(fh)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
